…input

TransformerBridge.forward() did not derive position_ids from a supplied
attention_mask, so left-padded input silently got the wrong absolute positions —
no error, no NaN, just wrong logits and a wrong loss. On gpt2 the loss for one
prompt moved from 4.503170 unpadded to 11.154946 with three left pads, while
HookedTransformer stays invariant (drift ~1e-06). Right padding was never
affected, since causality already protects it.

transformer_bridge.py derived position_ids only for batched *list* input, so
pre-tokenized tensors fell through to HF's plain arange and the padding offset
was never removed. This extends the same correction the list-input branch
already applies: when a mask is supplied, position_ids are absent, and the mask
indicates left padding, positions are computed from the mask. An explicitly
supplied position_ids still wins.

The bridge was also inconsistent with itself before this — the same batch gave
different logits depending on whether it was passed as strings or token IDs
(max |logit diff| 4.142e+01) — and enable_compatibility_mode(), which documents
"HookedTransformer-equivalent numerics", diverged on left-padded input while
matching exactly on unpadded input.

Adds integration regression tests covering logit invariance under both padding
sides, the same property in compatibility mode, and agreement between the
derived and an explicitly supplied position_ids. The right-padding cases are
controls: they pass with and without the fix. The tests live in the integration
tier because left padding makes a fully masked query row, which the Native
attention path turns into NaN until the masked-softmax fix in TransformerLensOrg#1608 lands.

Fixes TransformerLensOrg#1609.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…input

TransformerBridge.forward() did not derive position_ids from a supplied
attention_mask, so masked-out tokens silently shifted the absolute position of
every real token after them — no error, no NaN, just wrong logits and a wrong
loss. On gpt2 the loss for one prompt moved from 4.503170 unpadded to 11.154946
with three left pads, while HookedTransformer stays invariant (drift ~1e-06).
Right padding was never affected, since causality already protects it.

transformer_bridge.py derived position_ids only for batched *list* input, so
pre-tokenized tensors fell through to HF's plain arange and the offset was never
removed. This reuses utils.get_offset_position_ids — the same helper PosEmbed
and AbstractAttention already use — so the bridge shares HookedTransformer's
position derivation rather than paralleling it. An explicitly supplied
position_ids still wins, and an all-ones mask reduces to arange, so this is a
no-op when there is no padding.

The derivation is offset by any cached prefix: with past_key_values the mask
spans past+new while input_ids holds only the new tokens, so positions are
sliced back to the tokens actually being passed.

The bridge was also inconsistent with itself before this — the same batch gave
different logits depending on whether it was passed as strings or token IDs
(max |logit diff| 4.142e+01) — and enable_compatibility_mode(), which documents
"HookedTransformer-equivalent numerics", diverged on left-padded input while
matching exactly on unpadded input.

Adds integration regression tests: logit invariance under both padding sides,
the same property in compatibility mode, agreement with the shared helper's
derivation, precedence of an explicit position_ids, interior mask gaps, and a
cached decode step. Right-padding cases are controls that pass with and without
the fix. They live in the integration tier because left padding produces a fully
masked query row, which the Native attention path turns into NaN until the
masked-softmax fix in TransformerLensOrg#1608 lands.

Fixes TransformerLensOrg#1609.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…input

TransformerBridge.forward() did not derive position_ids from a supplied
attention_mask, so masked-out tokens silently shifted the absolute position of
every real token after them — no error, no NaN, just wrong logits and a wrong
loss. On gpt2 the loss for one prompt moved from 4.503170 unpadded to 11.154946
with three left pads, while HookedTransformer stays invariant (drift ~1e-06).

transformer_bridge.py derived position_ids only for batched *list* input, so
pre-tokenized tensors fell through to HF's plain arange and the offset was never
removed. This reuses utils.get_offset_position_ids — the same helper PosEmbed
and AbstractAttention already use — so the bridge shares HookedTransformer's
position derivation rather than paralleling it. An explicitly supplied
position_ids still wins.

The derivation fires only when the mask actually moves an attended token off its
default position, i.e. when some masked token precedes a real one. That covers
left padding and interior mask gaps. Pure right padding and all-ones masks
already agree with arange, so they are left alone: injecting position_ids there
is a no-op at best, and breaks models whose forward does not accept the argument
or which compute their own position streams (multimodal mRoPE).

With a KV cache the mask spans past+new while input_ids holds only the new
tokens, so the derived positions are sliced back to the tokens being passed.

The bridge was also inconsistent with itself before this — the same batch gave
different logits depending on whether it was passed as strings or token IDs
(max |logit diff| 4.142e+01) — and enable_compatibility_mode(), which documents
"HookedTransformer-equivalent numerics", diverged on left-padded input while
matching exactly on unpadded input.

Adds integration regression tests: logit invariance under both padding sides,
the same property in compatibility mode, agreement with the shared helper,
precedence of an explicit position_ids, interior mask gaps, a cached decode
step, and that no position_ids are injected when the mask does not require it.
Right-padding cases are controls that pass with and without the fix. They live
in the integration tier because left padding produces a fully masked query row,
which the Native attention path turns into NaN until the masked-softmax fix in
TransformerLensOrg#1608 lands.

Fixes TransformerLensOrg#1609.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>